Development of HMM-based acoustic laughter synthesis

Laughter is a key signal in human communication, conveying information about our emotional state but also providing social feedback to the conversational partners. With the development of more and more natural humancomputer interactions (with the help of embodied conversational agents, etc.), the need emerged to enable computers to understand and express emotions. In particular, to enhance human-computer interactions, talking machines should be able to laugh. Yet, compared to speech synthesis, acoustic laughter synthesis is an almost unexplored domain. Sundaram and Narayanan [5] modeled the laughter intensity rhythmic envelope with the equations governing an oscillating mass-spring and synthesized laughter vowels by Linear Prediction. This approach to laughter synthesis was interesting, but the produced laughs were judged as non-natural by listeners. Lasarcyk and Trouvain [3] compared laughs synthesized by an articulatory system (a 3D modeling of the vocal tract) and diphone concatenation. The articulatory system gave better results, but they were still evaluated as significantly less natural than human laughs. To improve laughter synthesis naturalness, we propose to use Hidden Markov Models (HMMs), which have proven efficient for speech synthesis. We opted for the HMM-based Speech Synthesis System (HTS) [4], as it is free and widely used in speech synthesis and research. The data used comes from the AVLaughterCycle database (AVLC), which contains around 1000 laughs from 24 subjects and includes phonetic transcriptions of the laughs [6]. HTS provides a demonstration canvas for speech synthesis, which enables to quickly obtain synthesis models with standard speech parameters. Our first works were to use this canvas to build a baseline for HMM-based laughter. Then, we looked at adapting our data and modifying some parts of the HTS demo to improve the quality of the obtained laughs. The major improvement of the AVLC database to better exploit the potential of HTS is the annotation of laughter “syllables”. This enables to include contextual parameters (e.g. the position of the “phoneme” within its “syllable”, the position of the current “syllable” within the current “word”, etc.) in the synthesis models.